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The free energy of the Random Energy Model at the transition point between ferromagnetic 
and spin glass phases is calculated. At this point, equivalent to the decoding error threshold in 
O l' optimal codes, free energy has finite size corrections proportional to the square root of the number 

of degrees. The response of the magnetization to the ferromagnetic couplings is maximal at the 
values of magnetization equal to half. We give several criteria of complexity and define different 
universality classes. According to our classification, at the lowest class of complexity are random 
graph, Markov Models and Hidden Markov Models. At the next level is Sherrington-Kirkpatrick 
spin glass, connected with neuron-network models. On a higher level are critical theories, spin glass 
phase of Random Energy Model, percolation, self organized criticality (SOC). The top level class 
O ' involves HOT design, error threshold in optimal coding, language, and, maybe, financial market. 

S Alive systems are also related with the last class. A concept of anti-resonance is suggested for the 
complex systems. 
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I. INTRODUCTION 



Complexity. The definition of statistical complexity is an entirely open problem in statistical mechanics (see 
[1,2] for the introduction of the problem and [3] for the recent discussion). There are a lot of different definitions 
having sometimes common context. A certain success was the discovery of an idea of "schema", a highly compressed 
information, introduced by Gell-Mann for complex adaptive (we assume, for all complex) systems. Some attempts, 
t— I \ based mainly on entropy concepts, have been undertaken to define the concept of complexity The approach [4-6], 
^ ■ relevant for our investigation, is of special interest. A very interesting aspect of complex phenomenon is related to 
the edge of chaos (the border between chaotic and deterministic motion), the phase of complex adaptive systems 
(CAS) [2], [7]. The concept of edge of chaos, independently suggested by P. Bak, S.A. Kauffman and C.Langton [8], is 
not well defined quantitatively. However, it is widely accepted this concept to be connected with the sandpile [9,10]. 
This concept is of special importance due to its possible relation to the birth of life and evolution [7]. This paper 
is devoted to relations between this phenomenon and some aspects of information theory and optimal coding [11]. 
We assume that the definition of a single (or best) complexity measure is a subjective one, even with a reasonable 
■ constraint that complexity should vanish for totally ordered or disordered motion (see dispute [12-13]). More strict 
is the definition of different universality classes of complexity, which is presented in the paper. We suggest several 
numerical criteria for complex adaptive property. In practice we suggest to identify the universality class of complexity 
from the experimental data and to choose a model from the same class to describe the phenomenon. 

We assume the following picture of complex phenomenon. The following hierarchy is presented: instead of micro- 
scopic motion of molecules or spins we deal with the macroscopic thermodynamic variables. Besides those, some new 
£j , structures arose, sometimes proportional to fractional degree of particles number. One can understand qualitatively 
the complexity as a measure of new structures. We are not going to scrutinize into the concrete feature of those 
structures. We will just evaluate total measure of structures on the basis of free energy expression, including finite 
size corrections. The suggested complexity measures could be applied for both pure systems, as well as to those 
defined via disorder ensemble (as in spin-glasses). For interesting cases of complex adaptive system, a hierarchy in the 
definition of a model, either a disorder ensemble (as in spin glasses), or scale of the system (spatial or temporal) should 
be represented. The structures themselves arc derived from microscopic motions of spins via order parameters. When 
those order parameter fluctuates, they can be handled like the microscopic spins or molecules. Therefore, in such 
cases, including the optimal coding of the article, we can identify complex phenomenon as a situation with changing 
reality or birth of new reality (the thermodynamic reality is a mapping of molecular motions into few thermodynamic 
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variables). A good analogy is the weak interaction in high energy physics on different scales. On the level low energies 
it can be described through a simple picture by E. Fermi. However on the level of 100GEV, when broken symmetry 
is restored, there is absolutely new reality. 

Our approach (to investigate just the ensemble averaged free energy instead of logarithm of the number of different 
ground states in spin-glasses [46]) is coherent with the idea of Jayenes [14] saying that there is a single probability in 
physics, measurable during observations, and there is no need to fracture it. 

We assume that it is proper to define the criteria of complexity via free energy than via the entropy (see the 
discussion in section three). In [15] a criterion of complexity, functioning from galaxies and stars to brains and society 
has been suggested: the rate of free energy density change. It is an interesting approach, but we prefer to deal with 
dimensionless criteria. The point that subdominant free energy describes the degree of system complexity has already 
been recognized, at least, for 2d critical systems. According to [16], the subdominant term is proportional to conformal 
charge (effective number of bosonic degrees of freedom). We just suggest to apply this criterion for any system, as 
one of complexity criteria. For the situation, when there is no explicit free energy (optimal coding, sandpilcs, etc.) 
one should try to find some equivalent statistical mechanical formulation of the theory and investigate free energy. In 
this work we give a first derivation of subdominant free energy for the case, related to an optimal coding and identify 
the universality class of error threshold. We provide other criteria of complexity which have not been considered yet, 
and the first list of universality classes. 

Optimal coding. Information processing should be certainly a property of complex adaptive system. What can 
be clarified by statistical physics? The connection of statistical physics to information theory has been known from 
well known works of Jayenes [14]. In 1989 N. Sourlas[17] found a connection of Random Energy Model (REM) of 
spin-glass by B. Derrida [18] with important branch of Shannon information theory, i.e. optimal coding theory [11]. 
In [19] I proved Sourlas idea for a principal case of finite velocity codes. An important result has been derived by P. 
Rujan [20] regarding to coding by statistical mechanics models at finite temperatures. In a series of work [21-25] we 
derived the main results of Shannon information theory using REM. Those results have been repeated by alternative 
methods later (see review [26]). Because we are going to consider models at the cross statistical mechanics-optimal 
coding, some features of optimal coding should be briefly mentioned [11]. 

Let us consider the transition of information, a sequence of ±1, through a noisy channel to receiver. There exists 
an original information which is a sequence of +1 and —1: t\ . . . cm- If we send some letter through the channel, due 
to the noise, the letters change their correct values, and information is partially lost. Therefore, to recover further 
the original message in a proper way it is needed to send originally more information using a coding. The encoding 
is a mapping of the initial message of length N onto a sequence which has the length aN (a > 1) 

(ei,...,ejv) -> (h(ei, ■ ■ ■ ,e N ), . . . f aN (ei, . . . ,e N )). (1) 

A noisy channel is represented as a mapping of the message by random letters r\j : 

(/l(ei, ■ ■ ■ ,£jv), • ••/ a! iv(ei, ■ ■ ■ ,£jv)) -> ■ ■ •>£iv)»7i, ■ --faN^i, ■ ■ ■ , £jv)?7qJv), (2) 

where the noisy rjj , 1 < j ' < aN are independent random numbers with probability distribution 

P(n) = l -^s [Vl - 1) + '-P^ + 1). (3) 

The transmitter introduces additional information (a > 1), and the receiver must extract useful information. These 
two operations are called encoding and decoding. In general, coding is a mapping of the initial message of length N 
onto a sequence which has the length aN, a > 1. Thus, encoding is done with aN functions fj = ±1. The value 
a^ 1 = R, the "rate" of information transmission, characterizes degree of redundancy Decoding in general case is 
called the procedure of extracting initial message out of noisy sequence (/i?7i, ■ ■ ■ , faNVaN)- 

When is the errorless decoding possible? We have a Boltzmann-Gibbs-Shannon measure of information for a discrete 
distribution pf. 



p l \np l . (4) 



In original message any letter ±1 carries an information In 2. In case of noise by Eq. @ any letter carries an 
information 

In 2 - h, 

h =-( o 111 o + o ln o )■ ( 5 ) 
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In the last expression we extracted from the In 2 the entropy h of the distribution by Eq. J3J • 

The encoding is possible to work over in different ways. To extract the original message without error it is reasonable 
to put a constraint 

aN[ln +I±Z^ l n 1±H« + IZH* In ±ZH*] > ATln 2. (6) 
1 2 2 2 2 1 ~ w 

On the left, we have information of the received message. On the right, we have an information to be extracted. 
This is Shannon fundamental theorem for errorless decoding. Only very special coding schemes correspond to special 
case, when the last expression transforms to equality. Such codes are optimal ones. They are universal mathematical 
constructions, like critical Hamiltonian in phase transitions. 

Statistical mechanics for coding. How could statistical mechanics be applied for the optimal coding? To encode 
the original sequence e\ . . . ejv, one constructs a Hamiltonian H(s), a function of (s± . . . sjv): 

- H(si . . . s N ) = /i(si . . . Sjv)/i(ei ■ • • £jv) H h f a N(si ■ ■ ■ Sjv)/cdv(ei ■ ■ ■ e N )] = h (yi, ■ --Vcin), 

Vj = fj( s i ■ ■ ■ sw)/j(ei ■ ■ • ejv)- (7) 

Here functions fj,l < j < aN, are products of some p spins, fj — Sj 1 . . . Sj , and H has a minimal value at Sj = rji. 
The influence of noise is very simple: every term (word) in JJJ is multiplied by a noise, and instead of pure Hamiltonian 
H(s) we have a noisy one 

- H(s,rj) = h (y[ . -.y' aN ), 
V'j = fj ( s i ■ ■ ■ s N )fj (ei • ■ ■ ■ (8) 

To find the minimum of the Hamiltonian, one could consider a statistical mechanics of the spin system with the 
Hamiltonian H at very low temperatures, 



z = E e-W'rt, (9) 



«i=±i 

where H(s, rf) = H{s\ . . . sjvj Vi ■ • ■ 7 7jv) ,/? — > oo. Without noise (rjj = 1) one can calculate the configuration (si . . . sjv) 
giving the main contribution to Z at (3 — > oo. We have the following expression for the mean magnetization: 

< s t >= e t . (10) 

It has been proved in [21] that Eq. (|ll)fl is correct also for the non-zero noise below the Shannon error threshold. 

In Shannon information theory one considers transmission of a message to a receiver. The influence of the noise 
corresponds to simple product of coding words fj(e% . . . ejv) by a noisy letter rjj (both are accepting the values ±1). 

Other versions of error threshold in statistical mechanics. What generalizations of the considered scheme 
are possible to accept? Instead of Eq. 10 one can consider a partition with the quantum noise: 

JV 

Z = Trexp{-/3[iT(<7f ...a z N ) + ^ of]}, (11) 

i=i 

and 

Z = Tr cxpl-^ 1 ^" i °*I^R{o\ . . . a z N )}}, (12) 
where H is a mean field like Hamiltonian like to 

i 

having minimum at configuration Sj = 1, 1 < i < N. The successful information transmission is connected with the 
phase, where < <rf >= m, > (there is a non-zero longitudinal magnetization), in Ea. (|ll|) quantum noise is additive, 
in Eq. ltT2"|l it is a multiplicative one. Eqs. (|11I12I) are connected to the evolution models [23, IH, |2j| when 
genetical information is transmitted to future generations. It is interesting that Eigen derived correct error threshold 
in its model [27] just from informational theoretical arguments long before the Sourlas work about connection of 
statistical mechanics with information theory. Eigen model has been exactly solved only recently |29j ] , Eigen's formula 
for error threshold was confirmed. 
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Our purpose is to connect the complex adaptive phase with the neighborhood of error threshold ©. We will 
consider the border between ferromagnetic and spin glass phases in Random Energy Model, investigating its statistical 
mechanics by Eq. We will not consider the informational-theoretical aspects of the problem any more-the subject 
is well discussed in [25]. In section 2 we will derive the finite size correction to free energy and investigate the 
dependence of magnetization from the bulk ferromagnetic coupling. In section 3 we will give a definition of complex 
adaptive property and define different universality classes. In section 4 we will suggest another concept of complex 
adaptive systems, i.e. the possibility of anti-resonance. In conclusion we will briefly discuss our results and general 
aspects of complex adaptive systems. 



II. RANDOM ENERGY MODEL 



Energy configuration formulation. To investigate the complex phenomenon we consider an equilibrium statis- 
tical physics situation similar to edge of chaos point i.e. the border between ferromagnetic and spin-glass (SG) phases 
in Random Energy Model (REM). The finite size corrections of free energy will be calculated later on. In REM N 
spins Si — ±1 interact through ( P N ) = p t^L p y ,P — * oo couplings with the Hamiltonian [18,21] 



H=~ £ {jl.^+jn.Ajs^.Si,. (14) 

l<ii..<i p <JV 



Here i are ferromagnetic couplings 



7° - M (15) 



and for quenched disorder we have a distribution 



Po(Jn., P ) = ^f^eM~jl, p %- (16) 

We see that there are ferromagnetic and random couplings, and Jo defines the ferromagnetic degree. 

In our spin model there are 2 N different energy configurations. It has been found by B. Derrida that for large values 
of p there is a factorization for energy level distribution. For a ^ (3 [18] 

p(E a ,E ) = p(E a )p(E f3 ). (17) 

For the 1-st configuration with = 1 [21]: 

p 1 {E 1 ) =-L=eM-(Ei + J N) 2 /N], (18) 

V7TiV 



and for other 2—1 levels [18] 



p(E) = ^=cxp(-E 2 /N). (19) 



REM has two equivalent definitions: via energy configuration Eqs. I|18ll9fl and via spin Hamiltonian version Eq. (14). 
It is possible to solve REM through ordinary spin glass approach, as well as using the factorization property Eq. (17). 
According to energy configuration approach, we perform averaging via energy level distribution (instead of random 
couplings in usual case of disordered systems): 

< InZ >=<ln£exp(-/3S a ) > E ■ (20) 

a. 

Here (3 is an inverse temperature. It is possible to derive [21] that at high enough values of Jo > Vhi 2, (see Eq. H27|) ~). 
at low temperatures the system is in ferromagnetic phase with magnetization 



m; = 1. 



(21) 
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Using the trick [18], 



OC 



<lnZ>=r'(l)+ / \ut d<C ^]- tZ]> dt. (22) 

at 



one can factorize the integration via different energy levels E a . The average is over energy distributions Eqs. (18), (19). 
It is enough only to calculate for < exp[— te (3Ei ] > for the single level. We consider 



1 f°° 

f(u) = —= I exp[-y 2 - e u exp(-\y)]dy, 

V 71 " J-oo 



(23) 



where A = 0\/N and < exp[— te l3Ei ] >— f(\nt). We can further derive for the < e tz >=< cxp[— te^«=i Ea ] > 

*(u) = [f(u + u f )f(u) M ], (24) 
where u = lnt,u f = J N/3, M = 2 N - 1. Now Eq. gives 

<lnZ>=r'(l)+ / u — — du. (25) 
J-oo du 

f(u) is a monotonic function. With exponential accuracy it equals 1 below 0, then becomes above it. We need four 
asymptotic regimes [18,21]: 

1 2u u 2 

f(u) « — r(^) e ^,A« u 
i r°° a 

f(u) w / dxe~ x ~ , | u |< A 2 

V 71 " /u/A 

/(.)«l-^r(-|)e^,4<,«-A 

/(«) « l- e "+^,-A 2 < U <-y. (26) 



We are interested in those asymptotic for u ~ iV or u ~ V^V and A ^> 1. As the f(u + uj)f(u) M is like to step 
function, its derivative is like to 6 function with center at some — uq. The vicinity of — uq contributes mainly to 
the integral in (|25J) (bulk value is equal to ug). Ferromagnetic phase appears, when the —Uf (the center of function 
f(u + Uf) is lefter, than — vJVAln2( the center of f(u) M ). The FM-SG border corresponds to: 

Jo = \/in2, oo > > Vhi2. (27) 

When there is only the first level with distribution {TSJ, < \nZ >= — /3 < E\ >= Uf = JoNfi. For that case 
\& = f(u + Uf). Therefore Eq. 1(25)1 gives: 



r'(l)+ / ud[/(« + «/)] =«/. (28) 

J — oo 

Using the last identity, we transform Eq. (|25l) into 

/OO /» CO pOC 

ud$(u)=Uf- ud$ 1 (u)=uf+ *i(«)d«, (29) 
-oo J — oo J — oo 

where *i(tt) = /(u + tt/)[l - f(u) M ]du. 

Exa ct b order of ferromagnetic and spin glass phases. Let us first consider the exact border of two phases 
Jo = vTn.2. $i(w) is a product of two monotonic functions, decreasing (one-to the left, another- to the right) from 
the point u = —ut. We define an auxiliary function F(u) by differential equation: 

F'{u) = f(u + u f ). (30) 
Using the second equation in (|26(l . we derive for | u |<C A 2 : 

\ /•oo 

F(u-u f )= dx^ e- y2 dy. (31) 
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Let us denote *&2( u ) = (1 — f(u) M ) and perform integration by parts in Eq. I|29l) : 

/oo /*oo 
F (u)^ 2 {u)du = it/ + F(cx))* 2 (oo) - F(-oo)* 2 (-oo) - / F(u)^' 2 (u)du 
- oo J —oo 

pOO 

= u f + [F(oo) - F(-u f )} - F'(-u f ) / (u + u f )*' 2 (u)du. (32) 



We have truncated expansion in degrees of u + Uf as ^(u) is similar to S function near the — uj. We used ^2(00) 
1, * 2 (-oo) = and F(-Uf) f™ x ^' 2 (u)du = F(-u f ). Then Eq. © gives: 



< InZ > -u f w / / exp(-y 2 )^ - TV*. (33) 



Eq. (|33|l is one of the main results of our investigation. It is obvious that besides the bulk term in free energy 
asymptotic there is a subdominant term proportional to square root of the number of degrees. The importance of 
the subdominant term in entropy has been underlined in [4], and has been well analyzed in [5]. They suggested to 
identify different universality classes of complex phenomena by the subdominant terms of entropy. In 1-d spin-glass 
model with long-range interaction, < J 2 ^ >~ — j) 2 , they derived Eq. (|33|) for the entropy. In [5] another object 
with a similar subdominant entropy has been mentioned, i.e. language [30]. 

Small deviation from the border of two phases. Consider small deviation from Eq. I|26|) (scaling is reasonable, 
as we see in Eq. (|3*5| ): 

Jo = \/In2 + 4^- (34) 
V N 

Now the finite size correction are less than in Eq.(33) and decrease exponentially at large values of jo: 

jo 



<\nZ > -(Vfo2 + J ^=)N - 0y/N exp[-jo]- ( 35 ) 
V N 



Now calculate the magnetization. We define 



"=< y^L >- (36) 



Using the identity -| = J °° die tz we derive for 



m 



d r°° d 

„, = -! dt-f(u + u f )f(u) M = 1- duf(u + u f ) — f M (u), (37) 



where u = Int. Using second equation in Eq. (26) we derive 

d < In Z > 



NNdjo 

And for its differential: 

dm 1 



1 f°° 



(38) 



rz = ^exp[-i 2 ]. (39) 
djo 



The last expression could be represented also as 

1 d 2 < In Z > I 



-pexp[-^]. (40) 



PVN d 2 j ^ 

Thus, at the exact border SG-FM (jo = 0) the dependence of magnetization from the external (ordered) parameter is 
maximal (maximum instability principle). This is likely a characteristic property of every complex adaptive system 
(CAS). One has an ordered external parameter to manage the system as well as random parameters (the choice 
of "ordered" and "random" could be subjective). There is an emergent (essentially collective) property. If the 
interaction with an environment is defined via the emergent property, then CAS drifts to the maximal instability 
point with maximal dependence of this emergent property from the ordered parameter. 
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One can consider the ^ as some degree of complexity. A close characteristic is the second derivative of free energy 
via ordered coupling, Eq. (|40ll . In our case, they coincide. However, there are possibly more complicated situations, 
when they are different and both should be used. 

Let us calculate the moments of P a = ^pC-/ 3 ^) Using the identity 

d n fo(t) 
dt n ' 

f (t) = /(lni). (41) 



1 f°° 

-= / exp[-y 2 - n\y - e u - Xy ]dy 



We have: 



^ /V - = / tdtMt) M -' d2fo{t d ^° N0) = I '—<i,. 



e 

Jo 



<Pl>= f 1 tdtMt)"- a ^-M*e J61t % 



< P a P-y > 



dt 



1 f°° 

< P a P~< > = 1 y= / exp[-x 2 ]dx. 

:.-»>! V ^ •/ — jo 



Q,7>1 



For the T <T C = 2Vhi2: 



^ < Pi > = [1 - -= / cxp[-x 2 ]dx][l 



T, 



Define 



^ poo rp 

V <P a P 1 > = [1--= exp[~x 2 ]dx] — . 

-A V 71 " J -jo 1 C 



(42) 



(43) 



C =< Pi > < p l > ■ ( 44 ) 

a>l 

C takes the maximal value at the critical point jo = 

T — T 

C=— . (45) 

2T C V ' 

For large jo we have that C decreases exponentially: 

C~exp[-j 2 ]. (46) 
The more detailed investigation of j = case states that 

< p i>=\,< p !>=\. (47) 

We see, that P\ — 0, 1 with probabilities 1/2. 

We can define C as edge of chaos parameter. At the exact error threshold border it has a maximal value, equal 
1/2 at zero temperature, i.e. the probabilities of ordered and random motions are equal. C decreases exponentially 
outside the region. What is the advantage of our choice Eq. (|4"4fl over another one, < pi > ^ a>1 < P a >? Eq. (44) 
distinguishes the — > oo as the most optimal situation, and the last choice fails. 

To define C, we have actually used the Tsallis entropy at q = 2 [32] 

_ _ ( 
q - 1 



In [3] M. Gell-Mann and S. Lloyd assumed the connection of I q with the edge of chaos systems. 
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III. DEFINITION OF COMPLEX ADAPTIVE PROPERTY AND UNIVERSALITY CLASSES 

Definition of complexity. Free energy is the fundamental object in statistical mechanics. The bulk free energy 
is proportional to the number of particles (spins). It is well known that in case of some defects on geometrical 
manifolds (lines, surfaces), besides the bulk term in the asymptote expression of free energy, there are subdominant 
terms proportional to some roots of N. Thus, the subdominant term in free energy could be identified with existence 
of some structures (much more involved than simple geometrical defects) in the system. In our case of REM, the 
formulation of the model was homogeneous in the space, but we got a square root subdominant term. In complex 
system we assume the following hierarchy: bulk motion and some structures above it. The subdominant free energy 
is related to the structures. If we are interested just in structure, we can ignore bulk free energy (an analogy in the 
physics of surface: to investigate the surface free energy we certainly miss the bulk energy). Therefore: 

A. We define the complexity as subdominant free energy. 

We have seen that in case of error threshold via REM it scales as a square root of number of spins. We assume that it 
is the most important class of complex phenomena, connected with alive systems. In complexity phase intermediate 
scale free energy (or entropy, or Kolmogorov complexity) becomes strong, and the subdominant term scales as a 
square root with the number of degrees. Eqs. . . 

What do we mean by intermediate scale? There is a minimal scale (ultraviolet cutoff) and maximal scale (infrared 
one). The intermediate scale is just their geometric average. In [33] has been investigated the statistics of the 
heartbeats. They found that healthy people can be differentiated by coarse-grained entropy at the intermediate scale, 
which is coherent with appearance of middle scale free energy in our case. Therefore the situation is coherent with 
the criterium A. The complexity in our definition is free energy on a higher hierarchy level (connected with the 
structures). One should remember that free energy itself is a second level on hierarchy. The energy is on the ground 
level is . Due to thermodynamic motion, only its smaller part is manageable on macroscopic level (only free energy 
could be extracted as a mechanical work while changing the global parameters). Therefore, complexity is a level on 
a hierarchy of the following modalities: energyfree energy and subdominant free energy ones. Every higher level is 
more universal. It is explicit in quantum field theory approach to critical phenomenon [16]. Different renormalization 
schemes can give different bulk free energies, but the same logarithmic subdominant one. Thus we observe a hierarchy 
of modalities (non-categorical statement about reality, see [34]). In principle, the hierarchy could be continued, and 
at some level the life could appear. Our view (rather statistical mechanical, than mathematical) is close to the one 
by M. Gell-Mann and S. Lloyd in [2], defining system complexity as "length of highly compressed description of its 
regularities" . 

Due to above mentioned hierarchy, the identification of complexity with a subdominant free energy is more universal, 
than the entropy approach of [4,5]. Sometimes the existence of structure could be identified in entropy or Kolmogorov 
complexity subdominant terms as well. In our case the free energy (sic!) reveals a huge subdominant term, but not 
the entropy. 

We assume that other features of our toy model are characteristic for complex adaptive systems: 

B. There is an emergent property, maximally unstable under the change of ordered external parameters, Ea. (|39|l . 
Sometimes it can be characterized as a second derivative of free energy via ordered parameter. 

C. The probability of ordered and disordered motions should be at the same level (like to Eq. (45), (47)). 

D. The complex adaptive properties could be exponentially damped in case of even small deviation of ordered 
parameter. 

Let us discuss different complex systems, defining universality classes. 

Critical theories. We assume that subdominant term of free energy describes the number of real parameters of 
the system. In [5] has been considered a learning process for a model with finite K parameters and a logarithmic 
subdominant term, proportional to K has been found. For 2d critical theories we can take either total effective number 
of bosonic degrees (conformal charge c), or the number of primary fields as a number of parameters. According to 
[16], nature has chosen the first one, and subdominant term of free energy is proportional to the conformal charge 
and to the logarithm of the degrees of freedom. 

We see that complex phenomena analyzed in previous sections correspond to another class of universality that the 
models in [16]. In critical theories [16] magnetization disappears at the transition point (contrary to error threshold 
case). Therefore, we admit that complex adaptive system, while having some scaling (fat tails in markets), could not 
be described by critical theories. 

In 2d percolation indices could be described by conformal field theory. Therefore, the percolation belongs to the 
class of critical theories. In our classification such situation is as complex as the class [16]. 

In spin glass model of REM B. Derrida found a logarithmic subdominant free energy. Therefore, the model belongs 
to the class of [16]. 

Financial markets. One can apply our criteria to financial markets [35]. To analyze the financial time series 
y{t) (US dollar-German mark exchange rate)the statistics of price increment y(t + t) — y{t) has been considered and 
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probability density function (pdf) p(x, r) has been constructed from the empiric data. A Fokker-Planck equation, 
where the role of time plays lnr, has been derived for the last distribution. We see a diffusion in the scale lnr as well 
as a drift. In [36] the ratio R of ordered motion of y and the diffusion has been calculated. It is the tail exponent of 
the y: P(y) ~ 5y~^ 1+>i ' [36]. In practice, R = /i ~ 3 — 5. In the situation, when the approach of [35-36] is correct, the 
more complex situation corresponds to the smaller values of /i. In case of error threshold model, considered in this 
paper, the subdominant term is larger in the region R ~ 1. Outside, it decreases exponentially like the one in Eq. 

For the markets something like the property can be also observed. There are fundamentalist traders who act in 
a deterministic way and the noisy ones [37]. In our model, they are similar to ordered and random couplings. In 
case of B, the fundamentalists' number is chosen to have a maximal influence to the market global characteristics. 
In usual thermodynamics we have a fundamental notion of temperature, and the equilibrium is possible only when 
temperature of different subsystems is the same. Now an edge of chaos parameter for the complex adaptive systems 
is introduced. It is reasonable to assume that in stable state it should be the same for different parts of the market 
(for example, for the traders and stocks). In this way it could be possible to predict future catastrophes. One can 
identify the edge of chaos parameters also considering a correlation matrix of different stocks. According to the above 
mentioned data, there are both deterministic and stochastic parts. It is very important to identify the subdominant 
term in entropy considering block entropies of financial data. 

Highly Optimized Tolerance (HOT) design. It is the last crucial achievement of complex system theory, 
related to the robustness of engineering design [38-39]. In the simplest case one considers fire forest model on 2-d 
lattice. There are trees at any site of lattice, and there is a known probability of sparks. As a tree is fired, its nearest 
neighbors are also fired. One constructs firebreaks (sites without trees) to limit the size of the event (total number 
of fired trees). The goal is to construct a robust scheme against fire propagation for the given spark probability, 
using a minimal area of firebreaks. Scaling laws for the distribution of fire events has been found. The situation 
highly resembles error threshold case. Actually, in [39] the connection of HOT design with source coding has been 
directly stated. In error threshold there is also scaling for the mean magnetization m = 1/2 + c/y/N [22]. It has been 
assumed in [38,39] that SOC and HOT design are different classes of universality. We can adduce another argument. 
In sandpile there is an analog of free energy, the number of recurrent states of the sandpile process. It is the number of 
spanning trees of free-fermions model. Therefore sandpile belongs to the class of universality [16] with central charge 
c = —2 [10]. We have used an important principle: the class of universality of the complex system should be the 
same in all of its representations. A very interesting feature of HOT design is that it gives a robustness against the 
originally given distribution of the noise. The robustness is very fragile: there is a large probability for the total crush 
(great fire) in case of the change of the original conditions. This resembles property D in the definition of complex 
adaptive phenomenon. In [40] has been suggested a constraint optimization with limited deviation (COLD) design 
to avoid large probability of total crush. They also mentioned the first known example of HOT design like situation. 
It is the classic problem of gambler's ruin: the optimizing total return leads to ruin with probability one [4l| . For a 
very complicated complex system with many hierarchies, the full optimization states a single simple principle for a 
management over the system, as in such case the essence of different hierarchies should be the same. Only the absolute 
optimization allows a full transformation of the content from one hierarchy level to another. This crucial feature has 
been lost in COLD case. I think that the choice of COLD can be successful only for not too much complicated systems. 
In the next section we will discuss a close concept of anti-resonance for the complex adaptive systems, exploring the 
property D in our approach. 

Markov models and random networks. There are a lot of applications of Markov models in complex systems. 
Especially important are applications in bioinformatics . There is some biological language in DNA and proteins, 
and hidden Markov models (the transition between states of the system is observed in a probabilistic way) have been 
applied to model this language. One can investigate the block entropies S(N) for the words with N letters in the 
stream of data and define the subdominant term. Such investigation has been thoroughly done in [6]. At large N, 
in case of classic order R Markov process S(N) gets an exact linear asymptote at N > R. For the case of hidden 
Markov model a subdominant entropy, decreasing exponentially with N has been found. This is very important 
moment. Those models, being very useful, don't share the class of universality of alive systems, which we assume as 
corresponding to the subdominant term ~ \/N. 

Networks are very popular in complex system research. How these geometrical objects could be classified into 
universality classes? In [43] has been introduced a statistical mechanics approach to describe the properties of a graph 
ensemble. The mean characteristics of the graph has been fixed, while maximizing the entropy of the ensemble. Now 
the number of pairs of vertices plays the role of number of degrees of freedom. The free energy can be defined. For the 
case of random graph there is no finite size correction in free energy expression. Therefore, random graph corresponds 
to the Markov model class of complexity. Unfortunately I don't see a way to enlarge the method of [43] to scale-free 
networks. 

Virus evolution near the error threshold. The evolution of the majority of viruses (RNA genome viruses) 
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is described well by Eigen model [27] . This brilliant model gives a simple and complete version of Darwin evolution 
theory. Information is represented here as a chain of spins taking A = 2 or A = 4 values. There are X N different 
configurations with corresponding probabilities pi,l < i < \ N . At any moment, the virus is giving offsprings 
with some rate specific for his genome (fitness). Offsprings randomly change their mother genome to other ones 
(mutations). When the majority of individuals has a genome near one configuration ("wild" one), then genetic 
information is successfully transferred to future generations. Otherwise, there is a flat distribution of individuals in 
the genome space. It is interesting that the virus evolution is near the error threshold. In " quasispecise theory" [44] 
(virus population with a distribution like a cloud around some "wild" genome configuration) there are equivalents of 
energy,i.e. fitness, free energyi.e. mean fitness for the whole system (for one configuration a product of fitness and 
errorless copying probability). All of these (selective abilities) can be derived in this model just as a consequence 
of Eigen equations. During the evolution, population is located mainly in a genomes with high selection ability. 
Considering the evolution in dynamic environments, it is possible to define a new kind of selective ability, like higher 
form of free energy (complexity?). Such approach to define a complexity is quite objective one. We assume that it 
is possible to calculate analytically also the ground state entropy (including the subdominant one) and define the 
complexity by [4-5]. 

It could be possible to investigate some aspects of optimal coding, impossible to do in alternative way. Choosing 
as REM's Hamiltonian like fitness function, we can get an analytical dynamics for optimal coding (the work is in 
progress). Thus, rigorous investigation of informational theoretical (complexity) aspects of evolution models could be 
very fruitful for both disciplines. 

Virus evolution is often referred to as a typical example of complex adaptive system. Another famous example is 
an immune system. Statistical mechanics has been successfully applied to this case [45] . I don't see a direct analogy 
with the error threshold phenomena here. But one should definitely choose model from high complexity class. 

Sherrington-Kirkpatrick model. Usually one defines the logarithm of different ground states [47] (solutions of 
Thouless- Anderson-Palmer equations) as a complexity. It is a reasonable characteristics to be investigated (although 
very complicated one). I think that to identify the universality class of the model it is enough to calculate finite 
size corrections of free energy or energy. Such calculations have been done for a Sherrington-Kirkpatrick model 
[48]. The subdominant energy scales as iV 1 / 3 . Therefore, it is a new class of complexity. For different spin-glasses 
other subdominant term scalings are possible as well, and finite dimensional spin-glasses are likely to have another 
universality class. We have mentioned the Sherrington-Kirkpatrick model [46] , because it is connected with neuron- 
networks. 



IV. ANTI-RESONANCE IN COMPLEX SYSTEMS 

Complex resonance. The concept of resonance is probably the most noticable phenomenon in nature, culture 
and science. The close notion of synchronization in complex system is becoming more and more popular [49]. We are 
going to analyze the idea of resonance in complex systems, to look for a possibility of, in some sense, inverse situation 
with an exponential damping of motion (anti-resonance). We suppose that this notion will compliment our view to 
complex systems in previous section. 

Originally, the simplest resonance situation has been investigated in mechanics of classical deterministic system 
with some resonance frequency, driven by external harmonic force. When two frequencies coincide, the reaction of the 
system to external force increases drastically. Even in this simple case we can observe two features of phenomenon. 
Frequency is an essence of motion, and there is a sharp peak in the ratio output-force. 

The next step was parametric resonance in classical mechanics. There is a hierarchy here. We observe a motion at 
given values of parameters, and the resonance frequency depends on the value of external parameters. If one changes 
the external parameter with the same frequency, as the frequency of the pendulum, there appears famous parametric 
resonance-the flow of energy from the higher level of hierarchy to the lower level one. Let us generalize this situation 
to other complex systems to define complex resonance. 

If there is a hierarchy in the system, and states at different hierarchic levels have some essence (comparable logically 
with each other), the generalized resonance happens, when these essences coincide. What about the essence of the 
state? In classical mechanics, there is only one real number characterizing total state, i.e. frequency. In general 
one should look for other total parameters of the system. In modern physics these are the following: temperature in 
statistical mechanics, replica system breaking scheme in spin-glasses (edge of chaos parameter), and the wave function 
phase in quantum mechanics. 

The next famous example of such (generalized parametric resonance) situation is related to the Nishimori line in 
disordered systems [50,51]. A hierarchy (quenched disorder) is present here. Sometimes it is possible to introduce 
some formal temperature to describe this disorder. If two temperatures (real one for the spins and the formal one for 
the quenched disorder) coincide, the system reveals some interesting properties becoming maximally analytic in some 
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sense. 

So we can define a hierarchy for the resonance. In the trivial case, the system is not hierarchic, it is logically 
homogeneous. The more involved case corresponds to the situation with principally different kinds of motions or 
(and) hierarchy. It is reasonable to define the second case as a complex resonance. In several situations (i.e. stochastic 
resonance), when it is impossible to define and compare clearly the essence of a state, one considers a situation, when 
there is a sharp peak in the ratio output-input at optimal value of external parameter. 

An important moment should be mentioned regarding our concept. If we consider some functional having different 
parameters, functions, logical structures, and we optimize it over the entire variables (besides some fixed group of 
parameters or functions) it could be stated that the essence of the whole system is the same, as that one of a fixed 
group. 

Anti-resonance. Let me now analyze the resonance situation with the opposite goal: to use the high levels of 
hierarchy to achieve a maximal negative effect. This is a situation not too rare in living systems. 
We define anti-resonance as a situation, when: 

1) a resonance is possible for some value of external parameter; 

2) it is possible to define the opposite phase transformation of the parameter; 

3) at the opposite phase values of the parameter there is either 

a) an exponential damping of a motion, or 

b) a new feature (opposite in some sense to those at the resonant parameter case) arose in a resonant way. 

The phenomenon is very complex. Thus, we are investigating the simplest models, trying to reveal those situations 
in complex systems, when such phenomenon is possible. Let us consider the pendulum with x(0) = xq,x'(0) = 0, 
when the frequency varies with some small amplitude h [54] : 

(fx 

— - = ~w 2 (l + hcoa(2wt + cj)))x. (49) 
dt z 

Here h <C 1, w is a frequency. Taking cos(2wi + </>)= sin(2u>i), we get an exponentially amplified solution solution: 

x{t) = exp(^i)[cos(wi)] (50) 
Choosing cos(2u>£ + </>) = — sm(2wt), we have an exponential damping 

x(t) =exp( — ^-t)[cos(wt)] (51) 
For the original amplitude A the damping period T is 

ilnA , . 

T~-j— . (52) 
lvw 

In this situation the picture is symmetric (both amplification and damping are possible). The other situation is 
possible with only resonant damping (like domino effect). 

Nishimori line. One considers [50,51] N spins Sj with interaction Hamiltonian 

H = — ^ t ji 1 ..i p Si 1 Si 2 ..Si p . (53) 
i\..i p 

There is a p-spin interaction here, couplings ji 1 ..i p are random quenched variables ±1 with probability for the 

values 1 and 1 ~ 2 "'° for the values —1. It is possible to write the following probability distribution: 

P(i,.,) = ^. (54) 
2cosh(po) 

The parameter (3o resembles an inverse temperature. Using the invariance of the Hamiltonian under transformation 

s i ► s iVi, jii_..i p ► ji 1 ..i p Vi 1 Vi 2 -- v i p i (55) 

in [50,51] has been calculated exact energy of the model at (3$ = (3. At (3 = (3$ our system has the best ferromagnetic 
properties in a sense that the number of up spins ^ s max i ma l & t Nishimori temperature [51]. In opposite 

phase, we can take (3 = — (3$. While the order parameter are different in ferromagnetic and antiferromagnetic phases, 
free energy is the same in both models, as Z(j,f3) — Z{j,—0) for the Hamiltonian (|53|) . For the odd values of p 
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one has an optimal properties for the configuration Sj = — 1. Thus, there is a trivial anti-resonance according to our 
definition. For the even values of p (i.e. p = 2) and bonds on the links of hypercube lattices in d -dimensional space, 
there is an antiferromagnetic ordering (an anti-resonance situation). 

Anti-resonance in complex systems. A search of anti-resonance in stochastic resonance [53] is a very interesting 
issue. The resonance is certainly a complex one, when the deterministic harmonic motion has the same period as 
the transition by noise. To construct the anti-resonance is problematic, as stochastic resonance has not a phase to 
reverse the resonance situation. In [54] the stochastic resonance explanation for the crashes and bubbles in financial 
markets (using the Ising spin model) has been considered. There is no phase for the noise to be reversed in stochastic 
resonance, but the information for the agents can certainly be positive or negative, thus moving the market from the 
border of two phases to one side. 

During the last decade, the idea of evolution or development at the edge of chaos [7-8], related to complex adaptive 
systems, was very popular. What about anti-resonance aspect of the origin of life? It is the case, at least, for 
the hyper cycle model by Eigen and Schuster [55]. One tries to construct self- replicating system overpassing simple 
problem, i.e. mutations, but, unavoidably, parasite creatures appear. As a result, there is a chance to consume all the 
information via those parasite creatures. We see, in some sense, a resonance picture with a chance for anti-resonance. 
The virus evolution is also often near the error threshold (mutation catastrophe) 56] . At the top level of life there is 
a phenomenon of apostasis, when the cell could be killed by a simple command. 

Our point of view is the following: even if complex systems are walking at the edge of chaos and climbing the 
mountains of fitness landscapes (in case of biological evolution), it is often a walk near the precipice. For the evolution 
it is not so dangerous, as only the survival of the species is crucial. One should be much more careful with a rare or 
single systems, like humanity. 

Complexity parameters and stability of complex systems. 

What parameters could be applied to analyze the complex systems? Besides the edge of chaos parameter, reasonable 
for the error threshold like systems, we can use Nishimori temperature like parameter. In principle Parisi's replica 
symmetry scheme also could be considered as a complexity parameter. Parisi's replica symmetry method works 
successfully for the mean field models, giving exact solutions [54]. For the Hamiltonian H(j, si) the ensemble average 
via quenched disorder j in n replica formalism brings to Hamiltonian Y^?—i H(j, s|) order parameters like q a ^ — 
5ZjSfsf/JV, where a, are replica indices. At the zero replica limit those parameters can be expressed via replica 
symmetry breaking parameters (given in our case by Eqs. (44), (45)). This scheme is a single for the whole system. 
Therefore, for the stability of the complex system it is important that those parameters coincide in different sub- 
systems. 

Very important problem is to look for complexity parameters in living matter. In case of proteins there is a notion of 
"design temperature", similar to the Nishimori temperature (see review [58]). Here Hamiltonian H(j,s) is a function 
of ji (amino-acids type in a sequence) and si (conformations). The couplings j have a distribution like the one in 
Eq.(54): 

P(j)~exp(-H(j,a)l3 d ). (56) 

where s is some ferromagnetic like "native" configuration. Perhaps the methodology of Nishimori line could be applied 
to the protein case. According to Nishimori [51] system with quenched disorder reveal best ferromagnetic properties 
at Nishimori temperature. It will be interesting to check this conjecture (Nishimori proved for the case of discrete 
spins) for the polymers. 

Besides the design temperature, well defined for the all proteins, there is also a notion of designability, which 
describes how robust is the ground state against the mutations. As a rule, the high designable proteins have also 
efficient low design temperature, but could be an exceptions (high design temperature with hogh design degree) [61]. 

In principle it is possible to construct equations for the protein evolution [59] as well as genome growth [60]. One 
can derive some degree of ferromagneticity for those cases as well. It is interesting to compare complexity parameters 
of those different aspects of living matter. 

V. CONCLUSION 

We have rigorously solved the error threshold for optimal codes using Random Energy Model, calculating magneti- 
zation and finite size corrections to free energy. This approach was applied in our previous works where many results 
of Shannon information theory about optimal coding were derived. There is an alternative method (replica approach 
with Nishimiori line), working well also in the case of realistic Low-Density Parity Check Codes (LDPC) [61] (see 
review [62]). REM approach could not be applied directly in the case of finite block coding, but it is much simpler. 
Main results of information theory were derived in REM approach about 6-9 years before those, found through al- 
ternative method: error threshold for finite rate of information transmission [19] versus [63], reliability exponent [25] 
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versus [64] , data compression [23] versus [64] . Multichannel coding was analyzed first in [24] . It is very interesting to 
check the universality class of codes with finite blocks length [61], optimal codes with finite number spin interaction. 
Unfortunately, an alternative method of [63-64] could not be applied here directly. 

Carefully investigating error threshold phenomena in REM, we have found several criteria of complexity: (33) and 
(35), (39), (44) and (46) which could be applied for complex adaptive systems. In [4-5] has already been suggested to 
consider the subdominant part of the entropy as a measure of complexity. We have enlarged their idea, suggesting 
to use a subdominant part of free energy as a measure of complexity. It is more universal, than the bulk free energy, 
and could be considered as the next step in the hierarchy energy-free energy-subdominant term in free energy. This 
hierarchy could be continued. Complexity appears on the third level, at some higher levels the life could appear. We 
admit that our approach catches the qualitative idea about edge of chaos: in the complex phase, the probabilities 
of ordered and disordered motions are equal Eq.(44), and complexity properties damp exponentially outside the 
error threshold point, Eqs (35), (39), (45). We adduced arguments that, unlike to SOC or ordinary critical theories, 
HOT design belongs to error threshold universality class of complexity. There are few classes of subdominant term 
behavior: zero, or exponentially decreasing subdominant terms for Markov, and Hidden Markov models [6]; logarithmic 
corrections for critical theories [16]; cubic root corrections for Sherrington-Kirkpatrick model; square root corrections 
for error threshold, long-range SG model [5] and, maybe, language. First a complexity class should be identified from 
the empirical data, to model complex phenomenon. As percolation or SOC models belong to the universality class 
[16], it is improbable that they can describe financial markets. Originally only SOC criticality has been identified 
with a qualitative idea of "edge of chaos". But we see that error threshold class is higher, than the SOC, and this 
complexity class is likely connected with alive-like systems [7]. We have introduced also the concept of anti-resonance, 
a phenomenon, perhaps, typical for the birth (and existence!) of life and for advanced complex adaptive systems. 

We have suggested to investigate, at first, the main features of complexity to identify the large universality classes. 
What other characteristics could be used for the further characterization of complex phenomenon? Perhaps the 
language of the system with its grammar, or, in physical systems, the existence of local gauge invariance. In case 
of REM, formulated as a spin model, there is a local gauge invariance (see Eq.(j2EJ)). There is local scale invariance 
for the models of [16]. Therefore two theories could be connected, according to our complexity analysis. It is really 
the case, as has been proved in [64]. We hope that other applications of such analysis are possible. The spin-glass 
phase and error threshold border in REM reveal the advantage of subdominant free energy approach to complexity 
compared with the subdominant entropy one. The latter, if be used as a complexity measure, produces lower classes 
(~ 0(1) instead of IniV or %/a"). We have used free energy to define the complexity. In general, when direct statistical 
mechanics formulation of the problem is impossible, one can use a variable, describing a manageable amount of motion 
on macroscopic level. The context of the problem can contribute greatly make a proper choice. For example, in Eigen 
model the equivalent of energy is fitness (with a minus sign). Free energy is automatically defined as a minus selective 
ability (mean fitness) of a group of configurations. 

In section 4 the idea of essence of the complex system state was used for several times. In case of spin glasses, the 
real state of the system is defined in the replica space, with some probability followed by the projection to the zero 
replicas (in Parisi's theory). In case of Hidden Markov Models the state is not directly observable again, as we get an 
information via probabilistic process. In quantitative linguistics, an abstract linear space has been applied to catch 
the meaning of the words [66]. Perhaps, the first example is quantum mechanics: there is a unitary evolution of the 
state in Hilbert space, and during the measurement we have some probabilistic results. In all those examples the 
state of complex system is not formulated directly via observable, but instead in some hidden abstract space, where 
the interpretation of the system (its motion) is rather simple one (the formulation of spin-glass statistical physics 
in replica space is much easier, than the zero replica limit, and formulation of Schrodinger equation is easier than 
quantum theory of measurement). We assume that it is an important feature of complex system: the real state of 
the system is in abstract hidden space, and can be observed in reality only in a probabilistic way. Therefore we 
suggest a " principle of expanded pre-reality" : to solve complex problem one should reformulate the problem in some 
internal, hidden, wider space ("pre-reality"), then return back to the observable space ("reality") in a probabilistic 
way. 

In lieu of the results, it is very important to look for anti-resonance phenomenon in stochastic resonance. Unfor- 
tunately, early attempts to find it have not been successful (H. Wio, private communication). Another important 
problem is to identify the universality class of turbulence. An accurate numerical analysis to identify the universality 
class (Y. Sinai, private communication) is likely possible for the case of Burgers turbulence. According to the whole 
experience of complex systems and our "pre-reality" principle, to succeed in turbulence solution one should formulate 
the problem in a wider abstract space, then return back to observable. It is very important to investigate the language 
models [66], and latent semantic analysis [67] in our approach. As we mentioned, the results of [31] (by means of 
entropy analysis) already supports the idea that language belongs to the error threshold class. The investigation of 
the semantic is much deeper. The singular value decomposition in [66-67] qualitatively resembles the fracturing of 
couplings into ferromagnetic and noisy ones. 
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